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Z~_~) • Abstract: A stratified sampling plan to audit health insurance claims is of- 

C ~ J ' fered. The stratification is by dollar amount of the claim. The plan is represen- 

' tative in the sense that with high probability for each stratum, the difference 

in the average dollar amount of the claim in the sample and the average dollar 
amount in the population, is "small." Several notions of "small" are presented. 
The plan then yields a relatively small total sample size with the property that 
■ the overall average dollar amount in the sample is close to the average dollar 

amount in the population. Three different estimators and corresponding lower 
confidence bounds for over (under) payments are studied. 
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1. Introduction 

Auditing health insurance claims is of necessity, extensively practiced. Statistical 
sampling plans and analysis of data from such plans is also extensive and diverse. 
One recent bibliography is Sampling for Financial and Internal Audits compiled by 
Yancey [B| . An annotated bibliography is given in Statistical Models and Analysis in 
Auditing (SMAA) [3[ compiled by the Panel on Nonstandard Mixtures of Distribu- 
tions, Committee on Applied and Theoretical Statistics of the Board on Mathemat- 
' ical Sciences, National Research Council. Most standard sampling methodologies 

. have been considered. Namely, simple random sampling, stratified random sam- 

pling, and dollar unit sampling. In addition a variety of estimators of overpayments 
qq . (underpayments) have been considered. (See SMAA j^.) These include the mean- 

per-unit estimator, the difference estimator, two types of ratio estimators, weighted 
t averages of the above three, dollar unit estimator based on estimating the pro- 

portion of items in which overpayment occurs and Stringer estimators based on 
an estimator of the above proportion and also the data corresponding to actual 
overpayments (underpayments). 

In this study we seek a stratified sampling plan whose main objective is to pro- 
duce a lower confidence bound for the amount of overpayments (underpayments). 
Stratification is done on dollar amount of the claim (book). It is envisioned that this 
lower bound could justify a repayment by a health care provider to a client whose 
employees are covered by the provider's health insurance plan. Desirable features 
sought include simplicity, representativeness, relatively small total sample size yet 
somewhat adequate sample size in each stratum, no samples from zero dollar claims 
and relatively larger samples from strata with high dollar claims. Furthermore the 
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plan should audit all extremely high dollar claims and treat them separately. That 
is, such claims should not be included as part of the statistical sample. 

To achieve the stated objectives we propose a "dollar representative stratified 
sampling plan" and refer to it as RepStrat sampling. The strata are class intervals 
of dollar amounts of claims. Class boundaries of strata are chosen according to 
typical guidelines (ample number of claims in each stratum, rounded numbers for 
boundaries, not too few, not too many). In addition the strata are adjustable so 
that the sample from each stratum is chosen in such a way that the average dollar 
amount of the claim for the population is "close" to the average dollar amount of 
the claim in the sample. The notion of "close" is made explicit in the next section. 
The closeness of the average dollar amounts of claims in sample and in population 
is what we mean by representativeness. Sample sizes for each stratum are chosen to 
ensure this closeness with high probability. We demonstrate that closeness within 
each stratum guarantees a higher degree of closeness between the averages in the 
overall population and overall sample. 

Representativeness, as measured by closeness of average dollar amounts in sample 
and population, is important for several reasons. First, it is intuitively desirable. 
Second, oftentimes estimates of overpayments are needed by an agency or com- 
pany in order to recover money that was excessively paid out. Since larger dollar 
amounts of claims have more opportunity for larger overpayments the agency would 
welcome higher average dollar amounts in the sample strata than in the popula- 
tion strata. On the other hand the agency's adversary would prefer smaller average 
dollar amounts in the sample. Since there is always a chance of litigation for the 
sake of recovering money, the least biased situation is to have representativeness in 
terms of the closeness notion. 

Representativeness is not the only feature of the plan proposed here. We want 
the sample size to be moderate. Not too large because of auditing expense and 
yet large enough to get an estimate of overpayment whose variance is not too 
large. Furthermore choosing samples within strata randomly allows for plausible 
estimators of means and variances in an unbiased way. Still further the plan offered 
here is not model based and does not require distributional or other assumptions. 
Thus the plan offered here is a balance of a sense of fairness in a litigation setting. 
Representativeness, randomness, distributional robustness, and adequate sample 
size. 

In Section 2 several different definitions of closeness will be offered and their 
properties will be studied. In Section 3 we will display customary formulas for lower 
confidence bounds based on several different estimators of overpayments. Section 4 
contains an example. Standard textbook references containing sections on stratified 
sampling are Cochran Scheaffer, Mendenhall, and Ott, 6th edition and Lohr 
0. 

2. Determination of strata and sample sizes 

Let N be the total number of claims in the population under study. Each claim has a 
dollar amount (book) and the first step in a stratified sampling plan is to determine 
L strata. The L strata will be formed as class intervals (Ai,Bi), i — 1,2, ... ,L. 
These L strata can be determined iteratively if necessary so that certain properties 
of the sample (prior to auditing) are achieved. Justification for stratification is an 
intuitive impression that higher book values are correlated with higher or greater 
likelihood of overpayments. In our case representativeness is easier to achieve with 
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stratification. In addition we want to assure that relatively more samples are drawn 
from higher dollar strata. 

Once strata boundaries are determined (not necessarily finalized) sample sizes for 
each stratum need to be determined. In general, the researcher seeks to estimate a 
population parameter (say fi) within a certain precision g, with a certain confidence 
level, I — a. In stratified sampling is the sample size for each stratum and 
the total sample size is n = Let Yij be the known dollar amount of 

the jth claim in the ith stratum, i = 1, . . . , L; j = 1, . . . , iVj, where iVj is the 
number of claims in stratum i. Let % = Yj/Ni and Vi = XX^y — Yi) 2 /Ni 

be the mean and variance respectively of the Yij in stratum i. Let and Xij be 
respectively the sample book amount and audited amount of the jth claim in the ith 
stratum, i = 1, . . . , L; j = 1, . . . , n^. Also let <£y = max(0, j/y- — x^) be the sample 
amount overpaid on an audited claim. The ith stratum mean and variance of dij 
are respectively denoted by \ii and af , and the mean of the overpayment variable 
for the population is denoted by fi where [i = Ni/j,i/N. Finally let Wi = Ni/N 
and Wi — rii/n. 

Note that when x^j ^ y^ that means the auditor has detected an error in treating 
the claim. A desirable or even acceptable error rate (that includes underpayments 
as well as overpayments) is 1%, which is the standard in some industries. However 
it is not uncommon to see higher rates of 3 to 5 or even 8 percent. In the latter cases 
overpayments or even overpayments minus underpayments can be in the millions 
of dollars. In Q the dij's are modeled as coming from a mixture distribution. One 
distribution of the mixture is degenerate at the point {0}. More discussion regarding 
the other distribution is given in [3j where it is mentioned that the distribution may 
depend on the book amount. In our study no distributional assumptions are made. 

There are a variety of ways sample sizes are determined or allocated to the strata. 
These include equal allocation (wi = proportional allocation (wi = Wi) and 

Neyman allocation (wi — Wi(Ti/ ^Wi<Ti). Neyman allocation is "optimal" in the 
sense of requiring the least total sample size to achieve the required overall precision. 
However knowledge of cr, is rarely available and proportional allocation is often 
preferred to Neyman allocation since it offers a type of "representativeness" in the 
sense that each stratum appears in the sample the same fraction of the time that it 
appears in the population. Researchers are sometimes willing to take a somewhat 
larger sample for "representativeness" and for the simplicity gained. 

In RepStrat sampling we allocate the sample to gain representativeness in the 
dollar amount of the claims in each stratum and in the overall population. Toward 
this end the researcher specifies a level of precision gi and a confidence level 1 — 7 
such that the sample is representative for each stratum in the sense that 

(2.1) P{\yi-Yi\ <9i) > 1-7, i = l,...,L. 

The practitioner may specify equal absolute precision in estimating stratum 
means, by choosing equal values for gi. This is considered in more detail under 
case (a) below. Alternatively, the practitioner may specify equal relative precision 
in estimating stratum means, i.e., gi — fYi, < / < 1; case (b) below deals with 
this case. Other choices for specifying stratum precision are considered in cases 
(c) through (e) below. We show that certain types of specifications are related to 
proportional allocation in stratified random sampling (case c), or Neyman optimal 
allocation (case d). We first deal with general {gi}. 

Given the practitioner specifies the desired stratum precisions in terms of the 
{gi} and the confidence 1 — 7, then the stratum sizes {n-i} can be determined 
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as follows. Strata boundaries are chosen so that the sample sizes determined will 
hopefully be adequate enough so that will be approximately normal. In light of 
this and (2.1) we find ni so that 




( 2 - 2 ) P\\Z\<9i\h7 TTT ITT > 1 ~ 7 



where Z is a standard normal variable. This leads to 

(2-3) n t = z^ViNi/tfiNi - 1) + z* /2 V-] , 

where z^/ 2 is the 1 — 7/2 percentile of a standard normal. 

Should Ni be large so that the finite population correction factor (fpc) be close 
to 1, then (2.3) reduces to 

(2-4) n t = z 2 l/2 Vi/gl 

Remark 2.1. Typically strata with larger dollar amounts of claims will also have 
larger values of V^. Thus from (2.3) we see that relatively larger samples will be 
drawn from such strata. This was one of the properties felt to be desirable in a 
sampling plan of this type. 

Now let y st — Nitji/N and Y = NiYi/N be respectively the sample estimate 
of and true population mean of book dollar amounts. Given the stratum {rii} are 
determined from the stratum precisions {gt} and 1— 7, and given the known stratum 
variances Vi of the Y^s yields the variance oiy st - We can use this to approximate the 
distribution oi\y st — Y\. For a specified value of g, we can approximate P{\y st — Y\ < 
g}; we will denote this probability by 1 — a. For a chosen value of g we consider 

(2.5) P{\y st - Y\ < g) = p{\y st -Y\ j ^ W ? V */ n * ^ 9 / 

If we ignore the fpc and substitute (2.4) in (2.5), we find that (2.5) is approxi- 
mately 



V 



(2.6) P||Z|<z 7/2 
Thus if 

L 

(2.7) Y. w "^/gf<i 



i=l 



it follows from (2.6) that the probability that the overall sample mean of claims is 
close to the population mean of claims is at least 1 — 7. That is, if (2.7) holds 

(2.8) P{\y s t~Y\<g} = l-a>l- 1 . 

Clearly in this instance a < 7. Note that (2.7) and (2.8) are easily satisfied in 
the case where gt — g, for all i; they arc also satisfied if gi — fY and g = fY 
for constant /. Thus, in the case where the stratum mean estimates precisions are 
equal (either absolutely or relative to the means), the combined stratum estimate 
has at least as good precision. 
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In the auditing application considered in this paper, claims are stratified by 
claim amount Yi into L strata. A special type of stratified random sample is picked 
with Hi claims from the iVj claims within the ith stratum, for i = 1, . . . , L. The 
stratum sample sizes are chosen to make the sample "representative" in that for 
every stratum, there is a high probability 1 — 7 that the stratum sample mean will 
be "close" (for the ith stratum, within gi) of the true stratum mean. Given the {gi} 
and 7, the {rn} are completely determined, as is P{\y s t — Y\ < 9}, for any g. The 
formula (2.1) through (2.8) detail this. 

In general stratified random sampling the desired closeness of y s t to Y is specified 
together with some type of allocation to determine the {rii}. This in turn determines 
the within stratum precision which is typically of minor or secondary interest. 
By contrast, the representative stratified random sampling approach specifies the 
desired "closeness" (of yt to Yi) within individual stratum to find the {rii}. In the 
next section we relate these two approaches to specifying sample sizes in stratified 
random sampling. 

3. Relation between representative and general stratified sampling 
approaches 

In general stratified random sampling, some particular method of allocating the 
total sample size n to the L individual strata is chosen. In equal allocation, rii = 
n/L. In proportional allocation, rii = nNij "^Nf, that is the rii are proportional to 
the Ni. 

How does representative stratified random sampling relate to various other types 
of allocation in stratified random sampling? To analyze this, fix the precision of the 
estimator y s t.'i that is specify g and a in equation (2.8). Various ways to specify 
the within stratum precision (the g^s and 7) are related to types of allocation 
(proportional, optimal and other) in stratified random sampling. For a given desired 
precision of the estimator y st given by a and g, and choice of spcification of gi, and 
7, we can find the total sample size n, as well as the sample weights u>i — rii/n. For 
example, we will show that if the gi are all taken to be equal, then Wi is proportional 
to the stratum variances, and the overall n is given by equation (3.4) below. More 
generally, we show how to determine {rii} and n given any three of {gi}, a, 7, g. 

Divide both the \y st — Y\ and g in (2.8) by \Jy^h=i WfVi/rii\ use the normal 
approximation and ignore the fpc, to find that approximately 



L 




Thus choosing m as in (2.4) yields 



(3.1) 




At this point we present a variety of choices of gi 

Case (a): gi — C for alii = 1, . . . , L. 
Then from (3.1) 



L 



(3.2) 
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and should we want rij to satisfy both (2.1) and (2.8), we have from (2.4) that 

L 

(3.3) n^V^l^Wf/g 2 . 

i=l 

That is, n, is proportional to Vi. The overall sample size in this case is 

(3-4) n=(z a/2 /gfj^WfY j V ] 

i=i j=i 

and the sample weights Wi — ni/n are given by 

L 

(3.5) w i = Vi/Y,V i . 

i=l 

Case (b): <?, = /Y^ for alH = 1, . . . , L. 
If is a fixed proportion of strata mean book amount, then from (3.1) we have 
approximately 



(3.6) 



^ i=l ' 



and thus if we want rn to satisfy both (2.1) and (2.8), we have from (2.4) 
(3-7) n^zl^jrw^yg^. 

Thus in this case m is proportional to Vi/Y 2 , which is the squared coefficient of 
variation of the book value for the ith situation. The sample weights Wi — rn jn arc 

(3.8) v./nf^Vi/n)- 

^ i=l ' 

Case (c): 9l = /^p. 
From (2.4) 

(3.9) = z 2 l/2 Wi/f = KW it 

when K is constant. This amounts to proportional allocation related to the number 
of claims in a stratum. This method of sample size determination docs not really 
satisfy our goals. 

Case (d): 9i = fVl /4 /Vm. 
From (2.4) 

(3.10) n i = z* /2 W iy /V i /f 2 . 
From (3.1) 



(3.H) f = gz l/2 /z a/2 ^w i vl 



/2 
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This is Neyman-optimum allocation for estimating mean claim size. For this case 
using (3.10) and (3.11) 



(3.12) 

and 

(3.13) 



X>^ /2 ) 2 4 /2 /V 
i=i ' 



w, = WiVi 



1/2 



1/2 



1=1 



Case (e): 9l = fY^ 2 . 
This designation of strata precision is a compromise between case (a) which seeks 
the same absolute precision for each stratum and case (b) which seeks the same 
relative precision. Here, using (2.4) and (3.1) 



/ = gZ~f/2/z a /2\fY^ W i Y ^ 



(3.14) 



m = Viz; 



: /2 E^7% 2 
j=i 



so 



(3.15) 



W; 



(Vi/Yd/^Vj/Yi. 

3 = 1 



Remark 3.1. In deriving most formulas for rti we have been ignoring the fpc. 
Should the fpc not be close to 1 a modification of the formulas may be necessary. The 
modification entails replacing Vi by Vi(Ni — rii)/(Ni — 1) and solving the resulting 
equation for m. So for example in case (a), (3.3) would be replaced by 



(3.16) 



L I r L 

3 = 1 ' L 3 = 1 



In case (b) the m of (3.7) would be replaced by 



(3.17) m 



^/2^(E^ 
3 = 1 



3 = 1 



Of course if we were concerned only with rn satisfying (2.1), recognizing the impli- 
cations this has for (2.8), then n, can be determined from (2.3) which includes the 
fpc. 



4. Estimating overpayments 

In this section we describe several point estimators and corresponding lower con- 
fidence bounds for (OP), the total amount overpaid. Most of the material here is 
essentially drawn from the cited SMAA Q article appearing in Statistical Science, 
and Cochran [l|. We focus on three of the estimators although the SMAA article 
discusses others as well. It would not be an unreasonable approach, when seeking 
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recovery of money, to give several estimates of (OP) to see if they agree. If they do 
not, the parties might choose to compromise among the competing estimators. 

We are somewhat guided in the selection of the three estimators by the findings 
of the SMAA committee in the sense that some of the following estimators lead to 
conservative lower bounds. The three estimators are 

The difference estimator: 

(4.1) {OP) d = Y,Nidi, 
where 

dj = dij/rii. 

3 = 1 

The separate ratio estimator: 

L 

(4.2) {OP)ns = Y, N & r i> 

i=l 

where 

n = di/m- 

The combined ratio estimator: 

L 

(4.3) (OP) RC =r c ^2N i Y i , 

i=i 

where 

L L 

(4.4) r c = Y,N i d i /Y,N i y i . 

i=l i=l 

Denote the sample variance of dkj by s 2 ,., where 

(4.5) 4 = , - dk) 2 /(m - l). 

Then the estimated variance of (OP)d is 

L 

(4.6) v(ap)d = N i( N * ~ n i>l/ n i- 

i=l 

A (1 — /?) lower confidence bound based on an estimate 9 for total overpayment is 

(4.7) e-z fj (V(§)) 1/2 . 

Note that V(OP)d depends on the s 2 d . which can be computed from (4.5), or in the 
case where many of the di's are zero, more simply from 



ni ]C d i3 ~ ( S ( X] } / n ^ Hi ~ 



3=1 3 = 1 J = l 
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where 

ni rii 

E d ^ and E 4 

can be computed from just the non-zero dij. This type of simplification is of even 
greater help for the ratio estimates that follow. 
The estimated variance of (OP) us is 



(4-8) V(OP) RS = E Ni'RSiW ~ ni)/m, 

where 



i=l 



s RSi = ^2( d a ~ nvaf/ini - l) 

(4-9) =4 +r 4 2 4 -2r,s diyi 



and 



S d,V, = E^ 4 - di)(Vi - Vi)/( n i - !) 



Note that and can be computed from J^dij, S4'i an d Yl^ioViji which 
can be computed from just the non-zero dij. Thus, just knowing (dij,yij) for the 
non-zero dij, and j/j and s 2 . are sufficient to compute (4.8). 

The estimated variance of (OP)rc is 

L 

(4.10) V(OP) RC =X^4W-^)M- 

i=l 

where 

s « = E^' ~ r cVijf/{ n i - !) 

E 4 yfi - 2r < E d ^ / - !) 

i=i j=i j=i J 7 

and r c is defined in (4.4). 

Knowing j/j and s 2 . gives X}j=i 2/5' an d knowing these together with the (dij,yij) 
for non-zero dij is sufficient to compute (4.10). 



5. Example 



We have constructed the following fictitious data set to be similar in number of 
claims, and book and audited amount of claims in data sets we have seen. Table 1 
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contains population data broken down by class intervals (strata) of designated dollar 
amount of claims. For each stratum we give the number of claims, the population 
mean {Y{) and variance (V^) of dollar amount of claim, and the sample sizes (nj) 
determined from (2.3) for 7 = .05, and gi = .05%. Table 2 contains sample data for 
those pairs f/ij) with positive overpayment, dij > 0. Note that there are several 
sample pairs (dij,yij) where yij — dij = Xij = 0. Such cases result when payments 
were made on claims that shouldn't have been paid. Table 3 contains the stratum 
sample means and variances for the dij and y^ . 

Table 4 compares the difference estimate (4.1), the separate ratio estimator (4.2), 
and the combined ratio estimate (4.4) for the total amount of overpayment. Also 
included are the 95% lower confidence bounds for total overpayment based on each 
estimator derived using (4.7). First we note that \y st -Y\ = |418.7500-417.9375| = 
0.812475 < 0.02Y. This indicates that RepStrat has accomplished one of its main 
goals. Namely that the average dollar amount of the claims in the sample is very 
close to the average dollar amount of claims in the population. 

From Table 4 we note that all three estimation procedures yield results that are 
very close to each other. In light of the SMAA report on conservativeness of the 
lower confidence bounds and the closeness of the three estimation procedures we 
feel that RepStrat has provided a satisfactory analysis of the data set. 



Table 1 

Population data and sample sizes 



Strata 


Dollar Value 


Number 


Population 


Population 




Number 


Strata 


of Claims 


Mean 


Variance 






Ni 


Yi 


Vi 


Hi 




(1) 


0-199 


4000 


120 


703 


74 


(2) 


200-499 


2200 


313 


3,500 


54 


(3) 


500-999 


1000 


620 


10,000 


39 


(4) 


1,000-1,999 


500 


1148 


30,000 


33 


(5) 


2,000-3,999 


200 


2374 


110,000 


27 


(6) 


4,000-6,999 


100 


5061 


250,000 


14 






8000 






241 



Table 2 

Sample Pairs: (dij,yij) when dij > 

Strata 

1 (9, 44), (105, 105), (57, 57), (143, 143) 

2 (8, 288), (422, 422), (115, 380), (93, 455), (495, 495), (359, 359) 

3 (530, 530), (76, 516), (12, 736), (124, 540), (54, 711), (96, 674) 

4 (804, 1804), (628, 1000), (718, 1000), (475, 1000), (500, 1500), (800, 1500) 

5 (1120, 2520), (2607, 2607), (389, 3456), (1990, 3265), (3900, 3900), (100, 3900), 
(1550, 3000), (500, 3000) 

6 (1220, 6102), (1750, 6999), (3, 5232), (6900, 6900), (100, 6671), (1220, 6102) 
height 



Table 3 
Sample Statistics 



Strata 


Hi 


Vi 


di 


s 2 

s y, 


1 


74 


115 


4.2432 


680 


2 


54 


300 


27.6296 


3400 


3 


39 


650 


22.8718 


10500 


4 


33 


1200 


118.9394 


30300 


5 


27 


2400 


450.2222 


111000 


6 


14 


5000 


799.5000 


250000 
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Table 4 

Estimators and Lower 95% Confidence Bound 



Type of Estimator 


Estimator 


Lower Confidence Bound 


Difference 


330,094 


214,037 


Separate Ratio 


329,833 


220,286 


Combined Ratio 


329,453 


215,323 
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